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INTRODUCTION 

The  present  study  represents  one  of  a  series  of  steps  taken  in 
New  York  City  to  establish  definite  goals  of  attainment  in  penmanship. 

On  May  7,  1914,  Dr.  William  H.  Maxwell,  City  Superintendent 
of  Schools,  appointed  a  committee  of  teachers  of  penmanship  to 
advise  the  Board  of  Superintendents  as  to  (1)  a  imiform  type  of 
letters  to  be  taught  in  all  the  schools;  and  (2)  to  formulate  instruc- 
tions to  teachers  as  to  the  teaching  of  muscular  movement  penman- 
ship. Subsequently  in  February,  1915,  the  Board  of  Education,  upon 
the  recommendation  of  the  Board  of  Superintendents,  adopted  a 
uniform  style  of  letters  to  be  taught  in  all  the  city  schools  and  also 
authorized  the  teaching  of  the  muscular  movement  method  of  writ- 
ing which  had  been  tried  out  in  a  number  of  schools  for  several  years 
previous. 

The  next  step  taken  was  the  construction  of  a  scale  from  the 
product  of  this  system  of  penmanship.  This  was  tmdertaken  at  the 
request  of  Acting  Superintendent  of  Schools  Straubenmuller  by 
Messrs.  Lister  and  Myers,  Instructors  in  the  Brooklyn  Training 
School  for  Teachers.  The  resulting  scale  was  officially  adopted  by 
the  Board  of  Education  in  February,  1918. 

The  setting  up  of  grade  norms  for  each  of  the  elements  measured 
by  the  scale  was  the  aim  of  a  study  undertaken  by  the  Bureau  of 
Reference,  Research  and  Statistics  in  June,  1919,  and  is  the  subject 
of  this  report. 

The  grade  standards  set  up  as  a  result  of  the  study  were  recently 
approved  by  the  Board  of  Superintendents  for  adoption  as  part  of 
the  course  of  study  and  syllabus  in  penmanship. 

Inasmuch  as  the  results  of  the  study  were  reported  upon  in  the 
December,  1920,  nimiber  of  the  Journal  of  Educational  Research, 
it  was  originally  decided  not  to  issue  the  report  as  a  publication  of 
the  Bureau.  Repeated  requests  for  the  report,  coupled  with  the 
adoption  of  the  standards  recommended,  have  led  to  its  publication 
at  this  time. 

Acknowledgment  is  herewith  made  of  the  assistance  rendered 
by  Mr.  Lister,  Instructor  of  Penmanship  in  the  Brooklyn  Training 
School  for  Teachers,  by  Mr.  Hugo  Newman,  Principal  of  the  New 
York  Training  School  for  Teachers,  and  by  Dr.  Hamilton  and  Miss 
Scheuerman,  Instructors  in  the  latter  named  school.  Without  their 
untiring  efforts  in  the  training  of  the  judges,  who  were  training  school 
students,  and  in  the  scoring  of  the  papers  the  siu*vey  would  have 
been  impossible. 


I.    THE  NEW  YORK  PENMANSHIP  SCALE 


1.  Characteristics 

The  New  York  City  Penmanship  Scale  possesses  certain  char- 
acteristics which  distinguish  it  from  the  scales  that  preceded  it.  In 
the  first  place  it  is  a  scale  constructed  from  the  product  of  the  sys- 
tem of  penmanship  it  is  intended  to  measure. 

The  second  characteristic  of  the  scale  is  the  fact  that  it  is  analy- 
tic. In  judging  ability  in  penman.ship  there  are  so  many  things  to 
be  considered  that  a  scale  based  on  general  merit  only  is  of  little 
assistance  to  the  average  teacher.  The  scale  should  help  the  teacher 
to  look  for  the  essentials  in  judging  a  given  specimen  of  writing. 
Legibility,  ease  in  writing  or  freedom  of  movement,  good  letter  forma- 
tion, uniformity  of  size,  uniformity  of  slant,  clear  and  uniform  spac- 
ing, neatness  and  an  easy,  fluent  appearance,  all  these  make  up  the 
value  of  writing.  *"A  scale  by  which  each  of  these  qualities  must  be 
estimated  or  judged  individually  would  be  too  complicated  to  be 
practical.  But  it  has  seemed  possible  and  practicable  to  group  most 
of  these  qualities  under  three  headings  and  thereby  make  a  scale 
that  will  enable  teachers  to  look  for  distinct  qualities  when  grading 
specimens  of  penmanship.  This  should  also  enable  the  teacher  to 
discover  where  the  penmanship  of  her  class  is  weak  and  to  point 
the  way  to  improvement;   in  other  words,  to  analyze  her  product." 

"With  this  idea  in  mind  it  was  decided  to  select  and  arrange 
specimens  under  three  distinct  headings.  Form,  Movement  and  Spac- 
ing, thereby  making  what  might  be  termed  a  triple  scale. 

"Under  the  caption  Form,  it  was  decided  to  consider  accuracy 
in  letter  formation,  with  the  standard  letter  forms  adopted  for  the 
schools  of  New  York  City  as  the  basis,  imiformity  of  size,  and  regu- 
larity of  correct  slant. 

"Under  Movement,  the  specimens  were  judged  as  to  quality 
of  line.  As  is  well  known  to  students  of  penmanship,  heavy,  tremu- 
lous and  broken  lines  are  evidence  of  either  finger  movement  or  mus- 
cular movement  that  is  too  slow  and  deliberate  to  produce  skill; 
and  clear-cut,  smooth  lines  are  evidence  of  the  use  of  the  easy-flowing 
muscular  movement  that  is  desired. 


♦Bulletin  No.  3,  Brooklyn   Training  School  for  Teachers,  The  New  York  City   Penmanship 
Scale,  by  C.  0.  Lister,  January,  1919. 


''Under  Spacing,  the  writing  was  judged  as  to  correct  and  uni- 
form spacing  between  letters,  between  parts  of  letters  and  between 
words." 

t 
2.    Construction  of  the  Scale 

*"The  scale  represents  the  average  judgment  of  twenty-one 
teachers  and  penmanship  experts  in  the  muscular  system  of  penman- 
ship, and  four  psychologists.  From  nine  schools  of  Greater  New 
York,  representative  of  the  best,  medium,  and  poorest  product  of  the 
muscular  system  of  penmanship,  3,550  specimens  were  selected 
from  at  least  one  entire  class  of  each  grade  from  3'B  to  8B  inclusive. 
Each  specimen  represented  one  trial  from  dictation." 

"The  specimens  from  each  grade  were  classified  into  four  piles 
by  the  writers  on  the  basis  of  general  merit  and  each  pile  was  thor- 
oughly mixed  with  its  corresponding  pile  of  the  several  grades.  Then 
on  the  basis  of  chance  three  hundred  specimens  were  selected  so  that 
practically  the  same  number  was  drawn  from  each  pile.  According 
to  written  instructions  each  of  the  twenty-five  judges  ranked  these 
specimens  in  eight 'piles  on  the  basis  of  equal  intervals  in  merit. 
Accordingly,  each  judge  ranked  the  three  hundred  specimens  three 
times,  namely,  as  to  form,  spacing  and  movement. 

"On  the  basis  of  the  average  rank  assigned  each  speqimen,  the 
best  and  the  poorest  were  selected  as  the  top  and  the  bottom  of 
the  scale.  Therefrom  the  exact  numerical  rank  which  the  other  six 
samples  should  have  was  determined.  The  specimens  whose  average 
ranks  are  the  same  as  these  determined  positions,  or  are  nearest 
them  were  selected.  Without  exception  all  the  samples  on  the  scale 
are  less  than  0.1  from  the  determined  position." 

Figure  1  illustrates  the  resulting  scale.  The  scale  consists  of 
twenty-four  specimens  arranged  in  parallel  columns,- — eight  under 
Form,  eight  under  Movement,  and  eight  under  Spacing.  The  scale 
values  of  the  specimens  under  each  element  are  90,  80,  70 — 20. 
Four  specimens  under  each  element  are  reproduced  below.  These 
represent  alternate  steps  beginning  with  90. 

*An  Analytic  Scale  of  Handwritiag.  by  0.  C.  Lister  and  G.  C.  Myers,  Journal  of  Educa- 
itonal  Psychology,  October,  1918. 
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II.    MEASURING  PRESENT  ACHIEVEMENTS  IN  PEN- 
MANSHIP 

As  a  first  step  in  the  determination  of  grade  standards  for  the 
New  York  Scale,  the  Bureau  undertook  to  measure  present  achieve- 
ments in  penmanship  in  the  schools.  Penmanship  tests  were  con- 
ducted in  fifteen  schools  selected  as  representative  of  the  whole  sys- 
tem.- 

1.  Conduct  of  Tests 

Uniformity  of  Conditions.  While  it  is  comparatively  simple  to 
secure  samples  of  the  children's  writing,  particular  care  was  taken  to 
have  uniform  conditions  obtain.  The  tests  were  given  by  thirty 
students  from  the  New  York  Training  School  for  Teachers  who  had 
had  the  coiu*se  in  educational  measurements  given  in  that  school 
and  also  the  normal  course  in  muscular  movement  penmanship. 
These  girls  were  familiar  with  the  various  scales  in  penmanship  and 
had  some  experience  in  the  conduct  of  standard  tests.  In  addition 
they  received  practice  in  giving  this  penmanship  test  in  accordance 
with  formulated  instructions,  considerable  attention  being  given  to 
the  question  of  "timing." 

The  children  in  the  classes  to  be  tested  memorized  on  the  day 
before  the  test  the  sentence  that  was  used,  viz.,  "One  must  exercise 
in  work  and  in  play."  In  the  lower  grades  the  class  teachers  gave 
attention  to  the  word  "exercise"  in  order  to  insture  familiarity  with  its 
spelling  by  each  child.  The  test  consisted  of  writing  repeatedly  the 
above-mentioned  sentence  for  two  minutes.  The  class  teachers  took 
no  part  in  the  testing,  although  present  in  the  rooms  at  the  time. 

The  test  was  given  in  500  classes,  involving  18,000  children,  in 
grades  4A  to  8B.  For  purposes  of  this  report  about  twelve  thousand 
specimens  were  scored  in  full. 

The  scoring  was  done  by  training  school  students.  The  methods 
employed,  the  training  of  the  judges  and  the  reliability  of  the  scor- 
ing that  resulted  are  discussed  in  a  later  section  of  the  report.  (See 
Appendix  A.) 

2.  General  Results 

Table  I  shows  the  median  scores  obtained  by  each  grade  group 
throughout  the  city  in  each  of  the  elements  measiu-ed  by  the  scale, 
form,  movement,  spacing  and  in  rate  of  writing. 
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TABLE  I— CITY-WIDE   MEDIAN   SCORES  BY   GRADES  FOR  FORM,   MOVEMENT, 

SPACING  AND  RATE 


FOBM 

MO'/EMENT 

Spacing 

Rate 

Grades* 

No.  of 

Pupils 

Median 

Median 

Median 

Median 

8B 

61 

62 

65 

82 

988 

8A 

57 

61 

63 

83 

1,117 

7B 

54 

56 

59 

79 

1,225 

7A 

55 

56 

59 

78 

1,327 

6B 

54 

56 

58 

72 

1,481 

6A 

51 

53 

56 

69 

1,414 

5B 

49 

49 

54 

69 

1,327 

5A 

45 

45 

51 

66 

1,529 

4B 

43 

43 

48 

63 

1,110 

4A 

38 

40 

48 

58 

24] 

City 

51.3 

52.9 

56.4 

72.5 

11,759 

*  The  letter  A  designates  the  first  half  of  the  school  year;   the  letter  B  the  second  half. 
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Figure  2  shows  the  above  graphically.  The  vertical  axis  shows 
the  rank  on  the  scale  and  also  indicates  the  rate  in  letters  per  min- 
ute. The  horizontal  axis  shows  the  grades.  For  the  city  as  a  whole 
the  median  score  for  all  the  pupils  tested  is  (without  regard  to  grade) 
51.3  in  form,  52.9  in  movement  and  56.4  in  spacing.  This  means 
that  50  percent,  of  the  pupils  tested,  without  regard  to  grade,  obtained 
these  scores  or  better.  The  table  also  shows  that  50  percent,  of  all 
the  pupils  wrote  at  a  rate  of  72.5  or  more  letters  per  minute. 
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In  order  to  show  concretely  the  product  of  the  schools  in  pen- 
manship the  following  specimens  of  pupils'  writing  are  presented. 
The  size  of  the  writing  is  somewhat  reduced  in  the  reproduction. 


FIGURE    3. 


SAMPLE    A,     ILLUSTRATING    THE    TYPICAL    PRODUCT    OF    THE 
BETTER   HALF    OF    THE    UPPER   EIGHTH    GRADE 


a 


^^-^^Q-'C-^rd--^ 


Form 

Movement 

Spacing 

Speed 

Sample  A 

70 
69 

70 
70 

70 
71 

92 

Q3 

94 

This  sample  was  equaled  or  surpassed  by  25  percent,  of  the  pupils  of  the  upper 
eighth  grade. 


FIGURE  4.    SAMPLE  B,  ILLUSTRATING  THE  TYPICAL  PRODUCT  OF  THE 
UPPER  EIGHTH  GRADE 


Form 

Movement 

Spacing 

Speed 

Sample  B 

Median 

60 
61 

60 
62 

65 
63 

82 
82 

This  sample  was  equaled  or  surpassed  by  50  percent,  of  the  SB  group. 
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Sample  A,  Figtire  3,  is  the  writing  of  an  8B  boy,  written  at  the 
rate  of  92  letters  per  minute  and  judged  to  be  of  quality  70  in  form, 
70  in  movement  and  70  in  spacing.  It  represents  approximately 
writing  that  was  equaled  or  surpassed  by  25  percent,  of  the  pupils 
in  the  8B  grade.  The  point  in  the  8B  distribution  which  includes 
25  percent,  of  the  pupils,  counted  from  the  upper  end,  is  called  the 
third  quartile.  The  third  quartile  is  located  at  68.8  in  form,  70  in 
movement,  71  in  spacing  and  94  in  speed. 

Sample  B,  Figure  4,  is  the  writing  of  an  8B  girl,  written  at  the 
rate  of  82  letters  per  minute  and  judged  to  be  of  quality  60  in  form, 
60  in  movement  and  65  in  spacing.  It  approximately  represents 
median  performance  for  the  8B  grade,  that  is  writing  which  was 
equaled  or  surpassed  by  50  percent,  of  the  8B  pupils.  The  median 
for  the  8B  is  61  in  form,  62  in  movement,  63  in  spacing  and  82  in 
speed. 

The  25  percent,  point  in  the  distribution  calculated  from  the  lower 
end  is  called  the  lower  quartile  or  first  quartile.  In  the  8B  grade 
■this  was  53  in  form,  53  in  movement,  58  in  spacing  and  68  in  speed. 
Sample  C,  figure  5,  represents  an  8B  pupil's  writing  which  approxi- 
mates these  values;  25  percent,  of  the  8B  pupils  wrote  as  poorly  or 
worse  than  this  sample  and  at  a  slower  rate. 

FIGURE    5.        SAMPLE    C,    ILLUSTRATING    THE    TYPICAL    PRODUCT    OF    THE 
POORER   HALF    OF    THE   UPPER   EIGHTH    GRADE 


Form 

Movement 

Spacing 

Speed 

Sample  C 

55 
53 

50 
53 

55 
53 

64 

Qi 

68 

This  sample  was  equaled  or  surpassed  by  75  percent,  of  the  pupils  of  the  upper 
eighth  grade. 

The  above  samples  may  be  used  also  to  illustrate  the  results 
obtained  in  the  other  grades. 
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3.  Rate  of  Writing 

Examining  the  data  more  in  detail  let  us  consider  first  the  rate 
of  writing.  The  specimens  of  penmanship  obtained  from  the  test 
were  scored  for  rate  of  writing  by  some  of  the  training  school  pupils 
who  had  served  as  examiners.  In  such  scoring  they  made  use  of 
mimeographed  copies  of  the  test  sentence  upon  which  there  was  a 
coimt  of  the  letters.  The  nimiber  of  letters  written  for  two  minutes 
thus  determined  was  reduced  to  the  rate  per  minute. 


TABLE  II— DISTRIBUTION  OF  INDIVIDUAL  SCORES  FOR  RATE   OF 
WRITING 


Letters 

G CADES 

Total 

Percent, 
of 

Cumu- 

per 

lat've 

minute 

4A 

4B 

5A 

5B 

6A 

6B 

7A 

7B 

8A 

8B 

total 

percent. 

150 

1 

5 

6 

0.05 

0.05 

140 

6 

6 

0.05 

0.1 

130 

1 



1 

2 

3 

2 

3 

12 

0.1 

0.2 

120 

3 

4 

6 

14 

18 

27 

21 

15 

108 

0.9 

1.1 

110 

1 

1 

2 

7 

8 

21 

32 

29 

53 

32 

186 

1.6 

2.7 

100 



13 

23 

32 

56 

101 

115 

98 

119 

92 

649 

5.5 

8.2 

90 

1 

41 

105 

116 

117 

172 

200 

219 

224 

188 

1,383 

11.8 

20.0 

80 

7 

69 

158 

148 

152 

181 

217 

210 

185 

192 

1,519 

12.9 

32.  y 

70 

38 

216 

341 

329 

325 

324 

367 

282 

256 

189 

2,667 

22.7 

65.6 

60 

63 

323 

363 

320 

327 

286 

223 

214 

149 

130 

2,398 

20.4 

76.0 

50 

51 

198 

221 

153 

186 

150 

99 

72 

58 

-  69 

1,257 

10.7 

86.7 

40 

61 

170 

178 

151 

177 

144 

39 

44 

40 

60 

1.064 

9.0 

95.7 

30 

16 

69 

101 

52 

49 

72 

12 

15 

10 

14 

410 

3.5 

99.2 

20 

3 

9 

28 

11 

8 

15 

3 

1 

3 

81 

0.7 

99.9 

10 

1 

6 

3 

2 



1 

13 

0.1 

100.0 

Total 

241 

1.110 

1,529 

1,327 

1,414 

1,481 

1,327 

1,225 

1,117 

988 

11,759 

100.0 

Table  II  gives  the  distribution  of  the  scores  of  all  pupils  by  grades. 
The  next  to  the  last  column  of  Table  II  shows  the  percent,  of  all 
pupils  that  wrote  at  each  rate  of  speed.  Such  data  reduced  to  a 
millenar}^  basis  in  figure  6  shows  that  out  of  every  thousand  pupils 
tested  1  wrote  at  the  rate  of  140  or  more  letters  per  minute,  1  at  130 
or  more  letters,  9  at  120,  16  at  110,  and  so  on  down  to  35  who  wrote 
less  than  40  letters,  7  less  than  30  and  1  less  than  20  letters  per 
minute. 

The  last  column  on  Table  II  shows  the  cumulative  percent, 
ranging  from  the  top  downward.  It  shows,  for  instance,  that  82 
out  of  every  thousand  or  8.2  percent,  wrote  100  or  more  letters,  that 
20  percent,  wrote  90  or  more,  that  32.9  percent,  wrote  80  or  more, 
and  55.6  percent,  wrote  more  than  70  letters  per  minute.  The 
standard  speed  set  up  by  Ayres  for  the  eighth  year  is  80  letters,  which 
is  equaled  or  exceeded  by  32.9  percent,  of  all  the  pupils.  Two  him- 
dred  pupils  out  of  every  thousand  or  20  percent,  equal  or  surpass  the 
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standard  rate  of  90  letters  set  up  by  Freeman  for  the  eighth  year. 
The  above  disregards  grade  distinctions  and  shows  that  the  New  York 
pupils  are  rapid  writers.  [^        _. 

. FiauRF        6 
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Table  III,  which  follows,  gives  the  median  rate  for  each  grade,  the 
rates  for  the  upper  quartiles  and  the  lower  quartiles  and  the  range  of 
the  middle  50  percent,  of  the  group.  It  also  gives,  for  comparative 
purposes,  standards  set  up  by  Ayres  and  by  Freeman  and  the  results 
obtained  in  the  Cleveland  Siu^ey. 

TABLE  III— GRADE  MEDIAN  RATES,  UPPER  AND  LOWER  QUARTILES 


Grades 

4B 

5A 

5B 

6A 

6B 

7A 

7B 

8A 

8B 

M 
Q^ 

73.0 
63.3 
51.4 

77.3 
66.3 
53.1 

79.3 
69.1 
57.5 

79.8 
68.7 
56.3 

86.7 
72.2 
59.3 

91.8 

77.8 
68.0 

93.7 
79.4 
6S.1 

95.4 
82.5 
70.9 

94.4 
81.5 
67.7 

Range 

Middle 

oOpercent. 

21.6 

24.2 

21.8 

23.5 

27.4 

23.8 

25.6 

24.5 

26.7 

Freeman 

Standards 

Ayrpa 

Standards 

Cleveland 

Survey- 

56 
56 

65 
64 
60 

72 
70 
70 

80 
76 
76 

90 
80 
80 
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Figure  7  shows  the  above  data  graphically.  It  appears  that  the 
median  rate  of  writing  increases  irregularly  from  grade  to  grade 
rising  from  63  letters  per  minute  in  the  4B  grade  to  82  in  the  second 
half  of  the  8th  year.  *  There  is  no  gain  shown  in  the  6A  grade  over 
the  5B  while  in  the  8B  the  rate  is  slightly  lower  than  that  of  the 
grade  below.     The  greatest  gain  appears  in  the  7A. 
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New  York  children  are  by  no  means  slow  writers.  Their  rates 
are  higher  than  the  scores  obtained  in  other  surveys  and  exceed  in 
all  years  the  standards  set  up  by  Ayres.  With  the  exception  of  the 
last  three  half  years  the  median  scores  are  higher  than  the  standards 
fixed  by  Freeman,  which  were  based  upon  the  average  of  the  better 
half  of  the  schools  tested  by  him. 


*  The  data  obtained  for  the  4A  grade  will  be  disregarded  because  of   the   small   number   of 
pupils  tested. 
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Figure  8  shows  by  grades  the  medians  for  each  class  in  rate  of 
writing.  Each  square  represents  a  class  and  the  figure  within  the 
square  is  the  median  score  attained  by  that  class.     The  scores  are 
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arranged  in  intervals  of  five.  For  instance,  in  the  top  figure,  the 
4B  grade,  there  were  three  classes  with  medians  of  50,  one  with  51, 
one  with  53,  and  so  on.  The  slowest  class  obtained  50  and  the  most 
rapid  78.  The  classes  in  this  grade  are  closely  bunched.  In  the  5A 
the  slowest  class  score  is  32  while  that  of  the  most  rapid  is  88,  more 
than  twice  as  large.  It  is  to  be  noted  that  the  majority  of  the  classes 
are  above  60. 

As  we  go  up  through  the  grades  the  central  tendency  moves 
toward  the  right  and  the  lower  scores  drop  off.  The  diagrams  show 
the  great  variability  in  performance  by  classes  within  the  same  grade, 
in  many  cases  within  the  same  school.  Grade  distinctions  have  little 
significance  when  one  considers  the  marked  overlapping  that  is 
revealed.  In  each  grade  quite  a  large  number  of  classes  could  be 
replaced  by  an  equal  number  of  classes  from  a  higher  or  from  a  lower 
grade  without  seriously  affecting  the  grade  distribution  as  far  as 
speed  or  rate  of  writing  is  concerned. 

It  is  to  be  borne  in  mind  that  we  are  dealing  not  with  scores  of 
individual  pupils  but  with  class  groups.  The  variability  cannot  be 
attributed  to  individual  differences  in  ability  but  must  be  charged 
to  lack  of  imiform  grade  standards  in  oiir  instruction.  While  such 
conditions  are  significant  they  are  not  surprising  and  have  been  shown 
in  a  number  of  studies  in  other  school  systems  jand  in  other  subjects 
besides  penmanship. 

Some  index  of  the  degree  to  which  the  grades  overlap  is  obtained 
by  comparisons  of  the  upper  quartiles  of  different  grades  with  the 
median  rate  of  higher  grades  (see  Table  III).  For  instance,  the 
upper  quartile  for  4B  is  73.0  which  is  a  higher  rate  than  the  median 
of  the  6B  grade,  72.2,  four  grades  above.  In  other  words,  25  percent, 
of  the  4B  children  write  faster  than  50  percent,  of  the  6B  children. 
Likewise^25  percent.,  at  least,  of  the  5A  children  write  a  good  deal 
faster  than  50  percent,  of  the  6B  grades;  25  percent,  of  the  children 
in  the  5B  attain  a  speed  of  79.3  and  exceed  the  rate  of  50  percent,  of 
the  7B  children,  and  25  percent,  of  the  6B  group  write  faster  than  50 
percent,  of  the  eighth  year  group. 

Comparing  the  lower  quartiles  of  different  grades  with  the 
median  of  lower  grades  we  see,  for  instance,  that  in  the  8B  grade 
25  percent,  of  the  pupils  write  more  slowly  than  50  percent,  of  the 
5B  grade,  six  grades  lower.  Similarly  25  percent,  of  the  6B  grade 
write  at  a  rate  which  fails  to  equal  the  median  of  the  4B  grade,  four 
grades  below. 

Reverting  to  the  grade  distributions  again  (Table  II)  which 
shows  the  individual  scores  in  each  grade,  we  see  even  greater  vari- 
ability between  the  rates  of  writing  of  children  within  the  same  grade. 
In  all  grades  practically  the  same  range  is  required  to  express  the 
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rates  obtained  by  the  individual  pupils.  In  the  4B  the  rate  varies 
from  that  of  one  pupil  who  wrote  less  than  20  letters  per  minute 
to  one  who  wrote  more  than  110  letters  per  minute.  In  the 
7B  grade  the  rates  range  from  that  of  the  slowest  pupil  who 
wrote  at  a  rate  of  less  than  30  up  to  five  pupils  who  exceeded 
150  letters. 

As  far  as  rate  of  writing  is  concerned  grade  distinctions  mean 
nothing.  To  say  a  pupil  is  in  the  8B  indicates  little  as  to  his  speed  of 
writing.  He  may  write  very  fast  or  he  may  write  at  a  slower  rate 
than  the  average  of  the  4B  pupils. 

The  condition  revealed  indicates  an  absolute  lack  of  grade 
standards.  It  does  not  necessarily  mean  that  emphasis  has  been 
put  upon  speed  but  rather  does  it  indicate  that  no  particular  atten- 
tion has  been  paid  to  controlling  the  rate  at  which  children  write. 
The  children  who  began  as  slow  writers  probably  continued  as  slow 
writers  throughout  the  grades,  with  only  such  increase  as  may 
come  from  increased  age  or  maturity.  The  fast  writers  in  the 
lower  grades  probably  continue  likewise  to  write  at  a  rapid  rate 
throughout. 

While  writing  of  a  good  quality  is  a  prime  requirement,  instruc- 
tion should  also  seek  to  improve  the  rate  of  writing,  particularly  of 
the  slower  writers.  Less  than  80  letters  per  minute  is  slow  writing. 
The  presence  of  n-umbers  of  pupils  in  the  last  year  of  the  course  who 
fail  to  attain  a  speed  of  60  letters  per  minute  indicates  the  failure 
of  OMT  instruction  to  give  pupils  about  to  leave  school  writing  habits 
which  will  meet  the  requirements  of  the  outside  world. .  The  presence 
of  pupils  in  all  grades  who  write  at  very  rapid  rates  again  indicates 
the  need  of  control  so  that  the  quality  of  the  writing  of  such  pupils 
is  not  sacrificed  for  the  sake  of  speed.  Rapid  writing  must  be  accom- 
panied by  satisfactory  quality. 


4.    Quality  of  Writing 

(a)    FORM 

How  well  do  New  York  pupils  write?  The  specimens  of  writing 
were  scored  for  form,  movement  and  spacing  by  means  of  the  New 
York  Penmanship  Scale. 

Under  "Form"  were  considered  the  letter  formation,  uniformity 
of  size  and  regularity  of  correct  slant. 

Table  IV  shows  for  form  the  medians,  the  upper  and  lower 
quartiles  and  the  range  of  the  middle  50  percent,  in  each  grade. 
Figure  9  shows  these  figures  graphically. 
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TABLE  IV— GRADE  MEDIANS,   UPPER  AND  LOWER  QUARTILES   IN  FORM 


Grade 

4A 

4B 

5A 

5B 

6A 

6B 

7A 

7B 

8A 

8B 

City 

Median 

47.1 
37.9 
30.0 

49.8 
43.2 
35.8 

53.4 
45.4 
38.3 

58.2 
49.2 
40.9 

59.5 
50.5 
42.4 

62.3 
53.7 
45.2 

63.6 
54.5 
45.6 

62.9 
54.0 
45.4 

67.8 
57.0 
47.7 

68.8 
60.8 
52.6 

Range  of  Mid- 
dle 50  percent. 

17.1 

14.0 

15.1 

17.3 

17.1 

17.1 

18.0 

17.5 

20. 

1 

16.2 

Grade  interval 
between  medians 

6.3 

2.2 

3.8 

1.3 

a 

.2 

0.8 

0.5 

3.0 

3.8 

The  grade  medians  for  form  show  a  sHght  irregular  progress  from 
grade  to  grade,  varying  from  37.9  in  4A  to  60.8  in  grade  8B. 
Improvement  is  shown  from  the  4A  up  to  the  6B.  In  the  6B,  7A 
and  7B  the  form  remains  practically  on  a  level.  In  the  8A  and  8B 
the  medians  show  a  gain.  The  average  grade  interval  is  but  2.5 
points  or  about  one-fourth  of  a  step  on  the  scale.  The  total  differ- 
ence between  the  4B  and  8B  medians  is  a  little  more  than  20  points 
or  two  steps  on  the  scale. 
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In  most  of  the  grades  the  range  of  the  middle  50  percent,  of  the 
group  approximates  the  total  median  improvement  between  the 
lowest  and  highest  grades.  In  other  words,  considering  only  the 
middle  50  percent,  of  the  children  we  find  there  is  approximately  as 
great  a  difference  in  any  grade  between  the  upper  and  lower  quartiles 
as  there  is  between  the  medians  of  the  4B  and  the  8B  grade. 

The  results  by  classes  in  each  grade  are  presented  in  Figure  10. 
Each  square  shows  a  class  and  the  figure  within  the  square  repre- 
sents its  median  score.  The  35  classes  tested  in  the  4B,  it  will  be 
seen  in  the  top  diagram,  range  from  30  for  the  lowest  up  to  57  for 
the  highest.  The  great  majority  of  classes  are  found  in  the  "forties." 
The  5A  grade  shows  greater  variation.  Here  the  range  is  from  33  up 
to  66.  The  class  with  the  lowest  score  does  only  half  as  well  as  the 
highest  class.     The  central  tendency  here  is  in  the  group  45  to  49. 
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The  advance  from  grade  to  grade  is  seen  b}^  the  gradual  shifting 
of  the  central  tendency  to  the  right,  that  is  toward  a  higher  score 
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and  also  by  the  falling  away  of  the  lower  scores  in  the  upper  grades. 
At  the  same  time  the  variation  in  class  performance  is  typical  of  each 
grade  and  the  overlapping  of  grades  is  quite  marked.  The  over- 
lapping of  grade  and  grade  may  be  roughly  indicated  by  a  compari- 
son of  the  upper  quartile  of  one  grade  with  the  medians  of  higher 
grades,  and  by  a  similar  comparison  of  the  lower  quartile  with  the 
medians  of  lower  grades.    Table  V  shows  such  a  comparison. 


TABLE  V— COMPARISON  OF  QUARTILES  WITH  MEDIANS  TO   SHOW   OVER- 
LAPPING 


Medians 

Lower 
Quartiles 

4A 

4B 

5A 

5B 

6A 

6B 

7A 

7B 

8A 

SB 

Upper 
Quartiles 

37  9 

43  2 

45  4 

49  2 

50  5 

53.7 

54.5 

54  0 

57.0 

60.8 

4A        30  0 

,,, 

2  3 

0.6 
4.2 

2  9 

4  5 
5.8 

3.7 
5.0 

J.S 

4-2 
5  5 
8  3 
.0 

1.2 
2  5 

5  3 

6  6 
5  9 

1.5 
1.5 
2.8 
2.1 
7  0 

47.1 
49.8 
53.4 
.58.2 
59.5 
62.3 
63.6 
62.9 
67.8 
68.8 

4A 

4B        .35.8 

—2.1 

4B 

5A        38 . 3 

— 4.9 
—2.3 

—  0.8 

5A 

5B        40.9 

—3  6 
—3  0 
—0.2 

5B 

6A        42.4 

-6,8 
—4  .? 
—3  6 
—3  8 
—1.5 

6A 

6B        45.2 

—3.6 
— 4  6 
—5  1 

—2.8 

6B 

7A        45.6 

—8  1 
—8  3 
-6.0 
— 1   1 

7A 

7B        45.4 

—9.1 
—6.8 
— 1   9 

7B 

8A        47.7 

—6.3 
— 1  4 

8A 

8B        52.6 

-^.4 

8B 

In  Table  V  the  medians  for  the  various  grades  are  given  across 
the  top,  the  lower  quartile  is  shown  in  the  left  hand  column,  and  the 
upper  quartile  in  the  last  colimm  on  the  right. 

The  figures  to  the  right  of  the  heavy  line  show  how  much  the 
upper  quartile  of  a  given  grade  exceeds  the  median  of  the  grades 
above.  The  figures  to  the  left  show  how  much  the  lower  quartile 
of  a  given  grade  falls  below  the  median  of  lower  grades.  For  instance, 
the  median  of  the  5B  grade  is  given  at  the  top  as  49.2,  the  lower 
quartile  is  given  at  the  left  as  40.9.  Reading  horizontally  across 
from  5B  to  the  left  of  the  heavy  line  shows  that  the  lower  25  percent, 
of  the  5B  groups  fails  to  reach  the  median  of  the  4B  grade  by  2.3 
points  and  the  median  of  the  5A  grade  by  3.6  points.  The  figiures 
on  the  same  line  to  the  right  of  the  heavy  line  show  that  the  upper 
quartile  of  the  5B  exceeds  the  6 A  median  by  7.7  points,  the  6B 
median,  two  grades  above,  by  4.5  points,  the  7A  median  by  3.7  points 
the  7B  median  by  4.2  points,  and  that  of  the  8A,  five  grades  above, 
by  1.2  points.  Similarly  for  the  other  grades.  In  the  8B  we  find 
that  the  poorest  writers  do  not  write  as  well  as  the  average  writers 
of  the  6B,  several  grades  below. 
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(b)    MOVEMENT 

Under  the  element  of  movement  was  considered  the  quahty  of 
line  on  the  assiimption  that  the  character  of  the  movement  would  be 
indicated  by  the  quality  of  line.  Table  VI  shows  for  each  grade  the 
medians,  the  upper  and  lower  quartiles  and  the  range  of  the  middle 
50  percent,  in  the  element  of  movement.  Figure  11  shows  these 
data  graphically. 


TABLE  VI— GRADE  MEDIANS,  UPPER  AND  LOWER  QUARTILES   IN 
MOVEMENT 


Grade 

4B 

5A 

5B 

6A 

6B 

7A 

7B 

8A 

8B 

City 

Q^ 

Median 

Q^ 

53.1 
43.3 
35.7 

54.6 
44.8 
36.4 

59.7 
49.3 
39.8 

63.3 
52.9 
43.1 

65.3 
56.1 
46.3 

65.6 
56.0 
45.9 

64.7 
55.7 
46.4 

69.9 
60.8 
49.2 

69.6 
62.0 
52.9 

63.5 
52.9 

42.2 

Range  of 
middle  50 
percent. 

VIA 

18.2 

19.9 

20.2 

19.0 

19.6 

18.3 

20.7 

16.7 

21.3 

Inter\ 
betwe 
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The  grade  medians  for  movement  show  an  irregular  progress 
from  grade  to  grade  rising  from  39.9  in  4A  to  62.0  in  8B.  The  extent 
of  improvement  is  slight.  In  the  7 A  and  7B  there  is  a  slight  loss 
which  is  made  up  by  the  gain  of  5  points  in  the  8A,  the  largest  grade 
increase.  The  average  gain  per  grade  is  but  2.5  points  or  one-fourth 
of  a  scale  step.  The  total  difference  between  the  4A  and  8B  medians 
is  but  22  points  or  a  little  more  than  two  steps  on  the  scale. 

Here  again,  as  in  form,  the  range  of  the  middle  50  percent,  of 
the  group  in  most  of  the  grades  is  almost  as  great  as  the  difference 
between  the  medians  of  the  highest  and  lowest  grades. 
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Figure  12  shows  by  grades  the  distribution  of  class  scores.  In 
the  4B  grade,  for  instance,  out  of  the  thirty-five  classes  tested  the 
lowest  class  received  the  score  of  29  and  the  highest  class  scored  53. 
The  greatest  number  of  classes  are  concentrated  in  the  forty  and 
forty-five  columns.     The  range  of  class  scores  is  quite  wide  in  all 
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grades.  In  the  8B  it  is  narrowest.  The  variability  in  the  perform- 
ance of  the  classes  and  the  overlapping  of  the  grades  are  readily 
apparent.  The  situation  in  movement  is  the  same  as  that  revealed 
in  form. 

A  comparison  of  the  upper  and  lower  quartiles  shows  likewise 
the  extent  to  which  grade  distinctions  are  without  significance  in 
this  subject.  The  upper  quartile  of  the  4B  exceeds  the  median  of  the 
6A,  three  grades  above.  The  upper  quartile  of  the  6A  exceeds  the 
median  of  the  8B  and  so  on.  On  the  other  hand,  the  lower  quartile 
of  the  8B  is  just  equal  to  the  median  of  the  6A  and  is  less  than  the 
median  scores  of  the  intervening  grades.  The  lower  quartile  of  the 
8A  is  lower  than  the  median  of  the  5B.  In  the  same  way 
the  lower  25  percent,  of  the  6 A  group  write  worse  than  50  percent, 
of  the  4B. 

(c)    SPACING 

Spacing  includes  the  elements  of  uniform  spacing  of  letters,  of 
parts  of  words  and  of  words.  Table  VII  gives  the  city  wide  median 
scores  for  each  grade  group  in  this  element.  It  also  shows  the  upper 
and  lower  quartiles  and  the  range  of  the  middle  50  percent. 


TABLE  VII— GRADE   MEDIANS,   UPPER  AND  LOWER  QUARTILES   IN   SPACING 


Grade 

4B 

5A 

5B 

6A 

6B 

7A 

7B 

8A 

8B 

City 

Q^ 

Median 

Q^ 

56.3 
47.7 
40.6 

58.4 
50.5 
42.2 

62.5 
54.0 
45.0 

64.6 
56.2 
48.2 

65.9 
57.7 
50.8 

66.9 
58.7 
51.0 

66.6 

58.7 
51.6 

70.7 
63.2 
54.3 

71.6 
64.8 
57.5 

65.5 
56-4 
47.5 

Range  of 
middle  50 
per  cent. 

15.7 

16.2 

17.5 

16.4 

15.1 

13.9 

15.0 

16.4 

14.1 

18.0 

Interval 
between 
grade 
medians . 


+2.8 


+3.5      +2.2 


+  1.5 


+  1.0 


0.0 


+4.5 


+1.6 


Figure  13  shows  the  same  data  graphically.  With  the  excep- 
tion of  the  7B  grade  the  medians  of  each  grade  show  an  increase, 
from  47.7  for  the  4B  to  64.8  for  the  8B.  The  total  increase  between 
the  4B  and  the  8B  grades  is  but  17  points,  less  than  two  steps  on  the 
scale.  This  is  approximated  by  the  range  in  scores  of  the  middle 
50  percent,  in  almost  each  grade.  The  average  grade  increment  is 
a  very  small  part  of  a  scale  step. 


29 


FmiiRF"SgL. 


4A 

B 

S*                  5B 

6A                   SB 

fA 

JPv 

^ 

BB 

■ 

■    ' 

1 

- 

'^ 

^■~ 

/ 

^ 

X 

/ 

^  - 

— 

^  ^  ^ 

MU  - 

^^'' 

. 

--i 

^'' 

' 

.- 

/ 

-         - 

"    -i           J 

z' 

^ 

, 

,?"_ 

/ 

V 

^^           _ 

, 

— 

— 

^  ^  -•  " 

«<' 

X 

^■''' 

y 

1     ^ 

\ 

'' 

/• 

-P 

'^ 

^^ 

^' 

^/ 

_ 

^ 

r  ^^ 

^ 

J' 

^■^ 

,^ 

^  "^ 

- 

^''               : 

V1eou»( 

3  Uppek 

QWRTILE 

tMf) 

^ 

Lowe  L 

^^ 

k 

^^ 

In!; 

1 

I 

_,/'' 

J 

^^ 

PACIN 

1    1 

s 

to—" 

1 

' 

1 

Inspection  of  the  quartile  and  the  median  scores,  as  with  the 
other  elements,  again  shows  what  Httle  significance  grade  distinctions 
possess.  The  upper  25  percent,  of  the  5A  group  do  better  than  the 
lower  25  percent,  of  the  8B  grade.  Fifty  percent,  of  the  8B  pupils 
do  poorer  than  25  percent,  of  the  6B.  Twenty-five  percent,  of  the 
8B  fail  to  get  as  high  scores  as  50  percent,  of  the  6B,  four  grades 
below. 

The  above  data  are  for  the  grade  groups  throughout  the  city. 
For  the  individual  classes  the  scores  are  shown  in  Figure  14.  As 
in  the  other  elements  the  marked  characteristic  is  the  variation 
between  classes,  in  many  cases  as  great  a  difference  being  present  as 
was  found  between  the  4B  and  the  8B  grade  groups  for  the  whole 
city.  Again,  also  the  overlapping  of  the  grades  shows  up  in  marked 
degree.     Such  data  are  class  scores  and  not  the  scores  of  individuals. 

4.    Relation  op  Speed  and  Quality 

The  previous  sections  of  the  report  considered  the  results  in 
terms  of  speed  and  of  the  elements  of  quality  alone.  A  very  import- 
ant question  to  be  considered  in  examining  the  results  is  the  relation 
between  quality  and  speed.  It  is  possible  for  us  to  set  up  grade 
norms  for  the  various  elements  of  quality,  norms  which  may  be 
achieved  but  at  the  expense  of  speed.  At  the  same  time  we  may 
establish   certain  norms   for   speed  which  also   may   be  attained 
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but  at  the  expense  of  good  form  or  other  characteristics  of  quaHty. 
Satisfactory  quality  must  be  attained  but  must  be  accompanied  by 
satisfactory  rates  of  v^n-iting. 

From  Table  II  which  showed  the  medians  in  each  characteristic 
of  quality  and  also  in  rate,  it  may  be  noted  that  speed  increased 
with  one  or  two  exceptions  from  grade  to  grade.  At  the  same  time 
the  median  scores  in  form,  in  movement  and  in  spacing  also  increased 
from  grade  to  grade.  There  was  a  certain  degree  of  correspondence 
as  far  as  the  whole  grade  groups  were  concerned  between  the  median 
speed  and  the  median  scores  in  quality  from  grade  to  grade. 

Does  such  relation  hold  within  the  grades?  In  order  to  deter- 
mine this  the  average  speed  was  determined  for  each  scale  score  in 
each  grade  in  form,  movement  and  spacing.  Table  VIII  shows  for 
each  grade  the  average  speed  at  which  each  quality  in  form  was 
written. 
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f^lCt-lRE.       XV 


AtE  SpEI- 

Letter.^ 


D  OfNA/r 

Per  Mini 


Figure  15  shows  graphically  for  each  grade  the  data  contained 
in  Table  XVII.  The  vertical  axis  represents  speed  in  letters  per 
minute.  The  horizontal  axis  represents  ascending  steps  in  form. 
From  the  diagram  it  will  be  noted  that  in  the  4B  grade,  for  instance, 
the  pupils  scoring  80  in  form  wrote  at  an  average  rate  of  48  letters  a 
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minute.  On  following  the  line  toward  the  left  it  is  seen  that  the 
rate  for  those  scoring  70  in  form  was  49  letters  per  minute,  for  those 
scoring  60  the  rate  on  the  average  was  59  letters,  for  those  scoring 
50  the  rate  was  62  letters  and  so  on.  Similarly  for  the  other 
grades. 

There  appears  to  be  a  general  tendency  toward  decrease 
of  speed  as  form  improved.  In  the  4B  grade  it  will  be  noticed,  how- 
ever, that  the  speed  averaged  fairly  constant  up  to  the  60  step  and 
then  declines  rapidly  for  the  upper  ranges  in  form.  Higher  quality 
evidently  is  attained  at  the  sacrifice  of  speed.  In  the  7th  and  8th  years 
also  the  speed  shows  quite  a  constant  average  from  50  steps  upward. 
The  higher  rates  of  speed  are  accompanied  by  lower  scores  in  form. 
Here  it  appears  that  the  greater  speed  results  in  poorer  form. 

Table  IX  shows  similar  data  for  the  element  of  movement  and 
is  illustrated  by  figure  16. 

The  same  general  tendency  toward  a  decrease  in  quality  with 
increased  rate  of  writing  is  revealed.  At  the  same  time  it  appears 
in  some  of  the  grades  that  with  the  intermediate  ranks  or  steps  in 
movement,  the  averages  for  speed  remain  fairly  constant.  For 
instance,  in  the  8B  grade  for  the  scale  step  from  30  to  80  the  rates  are 
85,  83,  81,  79,  80,  78.  In  the  8A  for  the  steps  from  40  to  70  in  move- 
ment the  rates  are  84,  84,  80,  83.  In  the  4B  also  the  rates  of  speed 
are  practically  the  same  for  all  ranks  on  the  scale  except  the  very 
extremes.  For  the  steps  from  30  to  60  inclusive  the  rates  are  64, 
63,  62,  60.  While  an  inverse  relationship  is  shown  it  does  not  appear 
to  be  very  large. 
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gure     XVI 


Table  X  and  Figure  17  show  similar  data  for  the  element  of 
spacing.  The  data  for  spacing  show  in  general  an  inverse  relation 
between  speed  and  spacing  with  an  intermediate  area  from  ranks  40 
to  60  where  no  correlation  is  apparent. 
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To  obtain  a  clear  and  more  exact  determination  of  the  relation 
existing  between  the  elements  of  speed  and  form,  movement  and 
spacing,  than  was  afforded  by  the  analysis  presented  in  the  preceding 
section,  the  degree  of  correlation  between  the  several  characteristics 
or  elements  was  computed.  Instead  of  treating  all  of  the  data 
involved  in  the  scores  of  the  12,000  pupils,  two  sets  of  1,000  specimens 
each  were  selected  by  a  process  of  sampling. 
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The  product  moment  formula  was  used  in  the  computation  of  the 
coefficient  of  correlation  with  the  following  results: 


TABLE  XI— COEFFICIENTS  OF  CORRELATIONS  BETWEEN  THE  SEVERAL 

ELEMENTS 


Elements 


1st  Sampling 


P.  E. 


2d  Sampling 


P.  E. 


Form  and  movement 

Form  and  spacing 

Movement  and  spacing. 

Speed  and  form. 

Speed  and  Movement- 
Speed  and  spacing 


+  .75 
+  .73 
+  .72 
—.056 

—  .036 

—  .013 


.0093 

.0094 

.0101 

.02 

.02 

.0213 


+  .749 
+  .738 
+  .716 

—  .054 

—  .029 

—  .016 


.0092 

.0097 

.0103 

.02 

.02 

.0213 


The  correlation  between  speed  and  the  elements  of  form,  move- 
ment and  spacing  is  negligible.  The  fact  that  the  probable  error 
is  as  large  or  larger  than  r  indicates  the  imreliability  of  our  result. 
It  appears  then  that  there  is  little  relation  between  speed  and  the 
different  elements  of  qualtiy  in  writing.  The  fast  writers  write  a 
good  hand  as  well  as  the  slow  writers.  The  good  writers  write 
rapidly  as  may  also  the  poor  writers. 

In  the  case  of  the  other  elements  a  high  degree  of  positive  corre- 
lation is  shown.  This  indicates  that  an  increase  in  form  would  be 
accompanied  by  an  increase  respectively  in  movement  and  in  spa- 
ing and  vice  versa. 


6.    Relative  Emphasis  on  Quality  and  Speed  in  Different 

Schools 

The  results  of  our  test  already  presented  have  clearly  indicated 
the  variability  of  achievement  in  the  various  classes  and  grades. 
It  has  been  shown  that  there  is  apparently  an  absolute  lack  of  uniform 
standards  of  achievement  in  the  various  grades  and  even  among 
classes  within  the  same  grade.  In  this  section  data  will  be  presented 
showing  the  relative  emphasis  placed  upon  speed  and  upon  quality 
in  some  of  the  schools  tested. 

In  order  to  show  the  development  in  penmanship  through  the 
grades,  the  progress  in  quality  and  the  progress  in  speed  may  be 
shown  under  one  diagram.     For  quality  the  grade  median  scores  in 
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form,  movement  and  spacing  were  averaged  to  obtain  a  single  score. 
The  resulting  curves  would  not  be  materially  different  if  we  were  to 
take  only  one  element  at  a  time. 
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Figure  18  shows  the  progress  from  grade  to  grade  in  three 
selected  schools.  The  horizontal  axis  shows  speed.  Distance  to 
the  right  indicates  progress  in  speed.  The  vertical  axis  measures 
quality  and  distance  upward  indicates  progress  in  quality.     In  inter- 
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preting  the  diagrams  we  must  bear  in  mind  that  the  distance  between 
the  horizontal  axes  measures  only  ten  points  or  one  step  on  the  pen- 
manship scale,  and  that  distance  between  the  vertical  axes  represents 
but  ten  letters. 

The  top  diagram  of  figure  18  represents  a  school  which  shows  con- 
siderable progress  in  quality  from  the  4B  to  the  6B.  The  score  in 
4B  in  quality  is  44,  in  the  5A  it  is  48,  in  5B,  56,  in  6A,  57,  and  in  6B, 
59.  At  the  same  time  it  is  apparent  that  no  effort  has  been  made  to 
control  the  speed  in  those  grades.  The  5A  grade  shows  a  loss  in 
speed  of  11,  the  5B  makes  a  large  gain. over  the  5A,  from  53  to  73. 
There  is  a  further  gain  in  the  6A  and  then  a  considerable  loss  in  the 
6B,  a  drop  from  77  to  63.  The  last  two  years  show  a  gain  in  speed 
but  with  the  exception  of  the  8B,  a  loss  in  quality  from  the  level 
reached  by  the  6B.  The  course  of  study  does  not  prescribe  penman- 
ship in  the  last  two  years.  This,  together  with  the  increased  demands 
in  the  upper  grades  for  rapid  writing,  may  explain  in  part  the  situa- 
tion revealed. 

The  middle  graph  in  Figure  18  shows  another  instance  of  irregu- 
lar development.  In  this  school  the  4B  scored  52  in  quality  and  72  in 
speed,  the  5A  scored  55  in  quality,  a  gain  of  3  points  but  only  63  in 
speed,  a  loss  of  nine  letters.  The  5B  shows  a  drop  in  quality  from  55 
to  45  but  a  gain  in  speed.  The  6A  makes  up  partly  for  the  loss  in 
quality  by  a  gain  of  8  points,  scoring  53  which  is  below  the  5A  score. 
The  6B  grade  score  shows  a  very  large  gain  in  speed  but  just  equals 
the  5A  in  quality.  From  the  4B  to  the  6B  the  net  result  is  a  gain  of 
only  3  points  in  quality.  From  the  7A  on  the  curve  turns  sharply 
upward  and  the.  apparent  emphasis  on  quality  results  in  a  con- 
siderable gain,  from  51  to  64,  but  is  accompanied  by  only  a  slight 
gain  in  speed.  The  8B  with  a  score  of  62  in  quality  and  88  in  speed 
shows  a  loss  in  both  but  still  reaches  the  8B  city  median  in  quality 
and  exceeds  it  in  speed. 

The  school  represented  in  the  diagram  at  the  bottom  of  Figure 
18  shows  an  erratic  course  of  development  and  indicates  the  lack  of 
grade  standards.  The  4B  and  5 A  grades  show  relatively  average 
quality  and  speed,  the  5B  and  6 A  show  an  increase  in  quality  with 
considerable  loss  in  speed,  the  6B  shows  an  equal  gain  in  quality 
and  speed.  The  last  two  years  gain  in  speed  but  fail  to  maintain  or 
gain  upon  the  level  in  quality  reached  by  the  6B . 

Figure  19  presents  graphs  for  two  other  schools  which  show  the 
different  emphasis  placed  upon  speed  and  upon  quality  in  the  differ- 
ent grades.  The  standards  of  the  teachers  in  the  different  grades 
within  the  same  school  are  most  variable.  It  indicates  that  the 
question  of  balanced  development  or  progress  in  both  speed  and 
quality  has  not  had  the  consideration  of  the  principal. 
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Figure  20  shows  three  additional  illustrations  of  differences  in 
relative  emphasis  upon  quality  and  speed.  The  middle  diagram 
shows  a  school  in  which  the  emphasis  has  been  placed  solely  upon 
quality  with  no  attempt  to  control  or  improve  the  rate  of  writing. 
Taking  quality  alone  for  instance  we  see  improvement  from  the  5A 
to  5B  from  46  to  51,  a  very  slight  loss  in  the  6A,  a  sharp  rise  in  the  6B 
to  57,  a  loss  in  the  7A  of  four  points,  a  gain  in  the  7B  of  five  points 
to  a  score  of  58,  a  further  gain  of  three  points  in  the  8 A  and  of  one 
intheSB. 
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SPEEK 
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All  the  grades  with  the  exception  of  the  7A  are  equal  to  or  above 
the  city  wide  medians  in  quality.  The  speed  shows  a  general  trend 
toward  the  left.  With  the  exception  of  the  7A  no  grade  equals  the 
5A,  the  lowest  grade  (in  this  school)  tested.  The  5A  with  a  speed  of 
71  exceeds  the  city  wide  5A  median  of  66  by  five  letters.  The6A 
equals  the  grade  median  but  all  the  other  grades  fall  below  their 
respective  city  medians.  The  school  needs  emphasis  in  speed,  partic- 
ularly in  the  upper  grades. 

The  data  presented  above  show  the  need  and  desirability  of  a 
study  of  grade  standards  in  the  several  schools  by  the  principals  and 
teachers. 

From  the  data  presented  it  is  quite  apparent  that  the  control 
of  the  schools  over  the  product  in  penmanship  is  anything  but  effec- 
tive. There  is  the  greatest  variability  in  achievement  in  the  various 
classes  of  the  same  grade  in  different  schools  or  between  classes  in 
the  same  grade  within  the  same  school.  The  median  improvement 
from  grade  to  grade  is  very  slight  and  indicates  how  futile  much  of 
the  instruction  must  be.  The  total  improvement  between  the  4B 
and  8B  grades  is  about  two  steps  on  the  scale  and  this  is  equaled  in 
most  of  the  grades  by  the  range  of  the  middle  50  percent,  of  the 
pupils.  The  progress  from  grade  to  grade  in  most  of  the  schools  is 
uncertain  and  erratic.  The  situation  is  such  that  you  can  predicate 
practically  nothing  about  the  penmanship  of  an  8B  pupil  or  even  of  an 
8B  class.  It  might  do  as  well  as  the  6B  city  average  or  it  might  not. 
The  SB  pupil  might  write  as  well  as  the  6B  grade  average  with  the 
speed  of  the  4B  or  even  vice  versa. 

The  situation  is  just  what  might  be  expected  from  a  lack  of 
standards  to  guide  the  teacher  in  her  instruction  or  the  principal  in 
his  supervision.  Without  the  control  which  standard  measurement 
yields  the  development  of  skill  in  penmanship  is  bound  to  be  erratic. 
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III.    TENTATIVE  GRADE  STANDARDS 

1.     The  Degree  oe  Penmanship  Ability  Found  in  The  Business 

World 

The  end  product  of  our  school  training  is  represented  by  the  8B 
pupil  who  is  about  to  graduate.  How  well  should  a  pupil  in  the  8B 
write?  At  what  speed?  Immediately  we  answer  such  questions  by 
asking  other  questions.  How  well  need  he  write?  How  fast  need 
he  write?  We  find  our  answers  in  the  demands  of  society  upon  the 
pupil  and  in  the  needs  of  his  later  life. 

There  have  been  a  number  of  studies  and  investigations  con- 
ducted to  determine  the  degree  of  writing  skill  required  by  the  out- 
side world  in  various  lines  of  endeavor.  Such  studies  however  were 
reported  in  terms  of  the  A3rres  scale  so  that  the  results  could  not  be 
used  in  setting  up  norms  for  the  New  York  scale.  An  independent 
attempt  was  made  to  determine  the  degree  of  skill  in  the  various  ele- 
ments of  handwriting,  as  measiured  by  the  New  York  scale,  that  was 
to  be  found  in  clerical  workers.  The  Bureau  obtained  under  test 
conditions  samples  of  handwriting  of  clerical  employees  in  several  of 
our  large  department  stores.  The  group  tested  numbered  161,  dis- 
tributed as  follows:  98  salesclerks,  17  billers,  39  checkers,  and  7 
clerks  and  bookkeepers.  In  all  these  positions  legible  rapid  writing 
was  considered  essential. 

Table  XII  presents  the  median  scores  in  each  element  for  each 
of  the  above  groups  in  comparison  with  the  median  scores  of  our 
8B  pupils. 

TABLE  XII— median   SCORES  OF  GROUPS  OF  CLERICAL  EMPLOYEES  IN 
DEPARTMENT  STORES 


Num- 
ber 

FOKM 

Movement 

Spacing 

Speed 

Group 

Med. 

Dif. 

from 
8B 

Med. 

Dif. 

from 
SB 

Med. 

Dif. 

from 

SB 

Med. 

Dif. 

from 
SB 

SB  Pupils 

988 

61 

62 

65 

82 

Sales.-  

98 
17 
39 

7 

53 
49 
58 
65 

—8 

—12 

—3 

+4 

68 
56 
66 
74 

+6 

—6 

+4 

+  12 

61 
59 
67 
63 

—4 
—6 
+2 
—2 

107 

128 

95 

99 

+25 

+46 
+  13 
+  17 

Billers _. 

Checkers 

Eutire  outside 

161 

54 

—7 

66 

+4 

63 

—2 

106 

+24 
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The  data  in  Table  XII  are  presented  in  figure  21  in  terms  of  the 
amounts  by  which  each  employee  group  exceeded  or  fell  below  the 
8B  medians. 
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In  form  the  salespeople  fell  8  points  below  the  SB  median,  the 
billers  12  points,  the  checkers  3  points.  The  bookkeepers  alone 
exceeded  the  8B  median,  to  the  extent  of  4  points.  In  movement, 
three  of  the  four  groups  exceeded  the  8B  median,  the  billers  falling 
below.  In  spacing,  the  outside  group  on  the  whole  fell  slightly  below, 
the  checkers  this  time  being  the  only  group  to  exceed  the  school 
median. 

In  speed  the  employee  group  greatly  exceed  the  rate  of  the  SB 
pupils,  the  salespeople  writing  at  the  rate  of  107  letters  per  minute 
or  25  letters  above  the  SB  median,  the  billers  12S  letters  or  36  above, 
the  checkers  95  or  13  above  and  the  bookkeepers  99  or  17  letters 
above  the  SB  rate. 
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The  data  presented  above  while  suggestive  are  not  conclusive. 
It  is  planned  to  continue  the  investigation  so  as  to  secure  more 
accurate  and  complete  data  as  to  the  demands  for  writing  skill  that 
are  presented  by  the  needs  of  the  outside  world. 

2.     Tentative  Standards  for  Each  Grade 

The  following  table  presents  the  tentative  grade  standards  pro- 
posed for  each  element.  They  are  shown  in  figures  22  and  23  in 
comparison  with  the  upper  quartiles  and  city  wide  medians. 

TABLE  XIII— tentative   STANDARDS  IN  PENMANSHIP  FOR  THE   NEW 
YORK  PENMANSHIP  SCALE 


Grade 

Form 

Movement 

Spacing 

Speed 

4B 

43 
47 
51 
54 
57 
59 
61 
63 
65 

46 
48 
50 
53 
56 
59 
62 
65 
67 

48 
51 
54 
57 
60 
63 
65 
67 
69 

56 

5A.__  

59 

5B 

63 

6A 

67 

6B 

7A 

71 
75 

7B 

80 

8A. 

8B.  __ 

85 
90 

For  the  8B  standard  in  form  the  average  of  the  scores  attained 
by  the  classes  which  exceeded  the  city  wide  grade  median  was 
taken.  This  score  of  65  is  already  reached  by  35  percent,  of  the  8B 
group  tested  or  by  10  of  the  33  classes  involved.  The  lower  end 
of  the  curve  was  begun  at  43,  the  city  wide  median  for  the  4B.  Th'fe 
cur\^e  of  the  grade  standards  was  raised  from  the  median  curve,  in 
some  places  arbitrarily,  and  in  other  places  through  smoothing  to 
the  points  indicated  In  the  6B,  7A,  and  7B,  the  grade  medians  were 
on  a  level  whereas  the  "standard"  curve  maintains  an  even  progress. 

The  same  procedure  was  followed  in  the  elements  of  movement 
and  spacing,  for  which  the  proposed  standards  are  presented  in 
Figures  22  and  23. 

For  speed  the  standards  adopted  are  taken  from  those  set  up  by 
Freeman.  Figure  23  shows  that  in  the  lower  grades  such  speed 
norms  are  below  the  grade  medians. 

The  above  standards  are  frankly  tentative.  When  the  achieve- 
ments of  the  better  schools  are  considered,  they  are  not  regarded  as 
imreasonable  nor  difficult  of  accomplishment.  The  schools  are  work- 
ing imder  a  uniform  time  allotment  and  what  is  accomplished  in  some 
of  the  schools  ought  to  be  attainable  elsewhere. 
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3.  The  Value  of  -Grade  Standards 
The  movement  for  standard  educational  measurements  has  for 
one  of  its  chief  purposes  the  setting  up  of  definite  aims  in  the  various 
subjects  of  the  curriculiim.  For  instance,  the  penmanship  in  our 
elementary  schools  aims  to  give  the  pupil  a  certain  degree  of  writing 
skill.  The  course  of  study  prescribes  that  seventy-five  minutes  a 
week  be  devoted  to  penmanship  in  the  lA  to  6B  grades  and  in  that 
time  the  child  be  trained  to  write  with  legibility,  ease  and  speed. 
Such  aims  are  vague  and  indefinite  and  therefore  afford  no  material 
guidance  to  the  teacher  and  the  principal  in  their  work  in  penmanship. 
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Contrast  the  aim  to  teach  an  8B  pupil  ''to  write  with  speed"  with  the 
aim  "to  write  at  the  rate  of  90  letters  a  minute."  Contrast  the  aim 
"to  write  with  legibility"  with  the  aim  "to  write  with  a  letter  forma- 
tion or  quality  equal  to  step  70  on  the  New  York  Scale."  The  use 
of  standards  provides  definite  teaching  aims. 

With  definite  aims  set  up  the  efforts  of  the  teacher  can  be 
directed  toward  the  specific  goal  instead  of  being  spread  over  an 
indefinite  field.  With  accurate  means  of  measurement  now  available 
the  teacher  can  readily  find  out  how  near  she  is  coming  toward  achiev- 
ing her  goal  and  upon  what  to  lay  stress  in  order  to  attain  her  aim. 
In  short  it  permits  the  teacher  to  plan  her  work  more  effectively. 

They  afford  a  basis  with  which  to  compare  the  present  accom- 
plishments of  the  pupils  and  thus  to  determine  which  pupils  are  below 
standard  or  above  standard. 

They  provide  definite  goals  for  the  pupils  also.  In  this  the  use  of 
standards  will  prove  an  effective  means  of  motivating  the  instruction. 
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They  provide  a  basis  for  making  more  effective  the  supervisory 
efforts  of  the  principal  and  the  superintendent.  Figure  24  shows  a 
graph  which  may  be  used  for  such  purpose.  The  graph  shows 
the  standards  in  fonn,  movement,  spacing  and  speed  for  each  grade. 
The  standards  for  the  4B,  for  instance,  are  43  in  form,  46  in  move- 
ment, 48  in  spacing  and  56  in  speed.  A  4B  class  that  was  standard 
in  achievement  in  all  elements  would  be  indicated  on  the  graph  by  a 
vertical  line  coinciding  with  the  vertical  line  under  4B . 

Figure  25  shows  the  use  of  this  graph  in  a  comparison  of  the 
achievements  of  SB  pupils  in  twelve  schools.  From  this  diagram  it  is 
possible  to  discover  the  elements  in  which  a  class  is  below  standard 
and  also  the  extent  to  which  it  is  below  standard.  The  graph  shows 
that  these  classes  are  nearer  the  8B  standard  in  speed  than  they  are 
in  the  other  elements.  It  shows  the  wide  variability  in  the  perform- 
ance of  different  schools.  While  some  are  above  standard  in  form 
most  of  the  twelve  schools  are  below,  several  in  fact  showing  but  sixth 
year  achievement.  The  superintendent  is  able  to  diagnose  the  situ- 
ation in  his  district  with  respect  to  penmanship  and  to  determine 
where  his  attention  should  be  directed. 
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Figiire  26  shows  the  achievements  of  five.  6B  classes  in  the  same 
school.  Ordinarily  it  would  be  reasonable  for  us  to  expect  that  the 
scores  of  such  classes  would  show  agreement  more  or  less  close.  The 
variability  is  apparent.  In  form  two  classes  are  below  standard, 
two  standard  and  one  above.  In  movement  one  is  below,  two  are 
quite  close  to  standard  and  two  are  above.  In  spacing  all  are  above. 
In  speed  there  are  three  above  and  two  below.  The  existing  situa- 
tion is  made  clear  to  the  principal  of  this  school  who  may  now  direct 
his  attention  more  effectively  where  it  is  most  needed. 
SuPFRVisoRY  Graph  5>Hf)wiMfi  Graof  .STANnAgns 


In  conclusion  attention  is  called  to  one  of  the  principles  under- 
lying all  educational  measurements,  and  that  is  the  principle  of  the 
limitation  of  training.  This  principle  means  that  as  soon  as  a  child 
has  surpassed  the  standard  of  his  grade  he  should  be  excused  from  any 
ftirther  direct  training  and  should  be  given  more  profitable  work. 
Particularly  should  this  apply  if  the  pupil  has  attained  the  skill  set 
up  as  the  ultimate  goal  of  the  elementary  school.  If  the  needs  of  the 
outside  world  have  been  a  factor  in  determining  such  standards,  as 
they  should  be,  any  further  development  of  skill  beyond  such  stand- 
ards will  not  probably  be  a  factor  in  determining  the  social  efficiency 
of  the  individual.  The  development  of  higher  levels  of  ability  in 
preparation  for  a  specific  vocation  is  not  the  function  of  the  elementary 
schools.  Again,  individuals  with  greater  natural  capacities  will 
reach  the  standard  level  with  much  less  training  than  individuals  less 
favored  by  natiu-al  inheritance,  and  consequently  for  such  individuals 
drill  soon  comes  to  have  little  meaning  and  in  fact  may  be  harmful. 
For  the  exceptionally  able  child  the  incidental  training  which  comes 
through  the  use  of  his  mechanical  skill  in  real  work,  once  the  standard 
has  been  reached,  will  automatically  develop  his  ability  to  higher 
levels  without  direct  drill.  The  ordinary  child,  on  the  other  hand, 
may  probably  require  continuous  direct  drill  to  maintain  his  level. 
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APPENDICES 


APPENDIX  A 
Methods  of  Scoring  Employed  in  This  Survey 

The  scoring  of  the  resulting  twelve  thousand  samples  of  penman- 
ship in  each  of  the  elements  of  form,  movement,  spacing  and  speed, 
represented  an  enormous  task.  The  task  was  undertaken  by  Dr. 
Hamilton  and  Miss  Scheuerman  of  the  New  York  Training  School 
for  Teachers  and  by  Mr.  Lister  of  the  Brooklyn  Training  School  for 
Teachers.    The  services  of  training  school  students  were  enlisted. 

While  it  is  generally  conceded  that  the  use  of  a  penmanship  scale 
in  the  hands  of  trained  scorers  reduces  considerably  the  variability 
in  judgment  which  obtains  when  specimens  are  scored  without  such 
aid,  it  is  also  true  that  in  the  hands  of  untrained  persons  the  penman- 
ship scale  gives  no  guarantee  that  the  resulting  judgments  will  be 
reliable.  Nor  does  a  knowledge  of  the  theories  underlying  educational 
measurements  or  the  statistical  derivation  of  scales  insure  acciurate 
scoring.  Likewise  expertness  in  teaching  penmanship,  while  of  some 
help,  is  not  synonymous  with  reliability  in  scoring.  To  obtain  accu- 
racy in  scoring  there  must  be  training  in  the  use  of  the  scale. 

Investigations  by  Kelly  and  others  have  shown  that  teachers 
using  the  percent,  method  of  rating  have  reduced  their  variability  by 
practice  at  the  expense  of  the  children,  while  they  have  at  the  same 
time  decreased  their  capacity  for  effective  use  of  a  standard  scale. 

It  was  considered  advisable  for  the  above  and  for  other  reasons 
to  employ  as  scorers  training  school  students  rather  than  the  teachers. 
The  training  school  students  who  acted  as  judges  had  the  following 
qualifications. — (1)  They  had  been  instructed  in  the  muscular  move- 
ment method  of  writing.  (2)  They  had  taken  the  normal  course  in 
teaching  that  method.  (3)  They  had  also  had  the  course  in  educa- 
tional measurements  in  the  training  school  as  a  result  of  which  they 
were  familiar  with  the  various  penmanship  scales  and  in  the  course 
of  which  they  had  received  some  practice  in  giving  standard  tests  and 
in  judging  specimens  of  penmanship  by  means  of  standard  scales. 

To  supplement  the  above  favorable  conditions,  it  was  arranged 
to  give  the  students,  who  were  to  act  as  scorers,  specific  instruction 
and  training  for  a  period  of  three  weeks  or  longer  in  the  use  of  the 
New  York  Scale  through  actual  practice  in  judging  specimens.. 
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For  use  in  the  training  of  the  judges  standard  specimens 
of  writing  were  prepared,  that  is,  the  specimens  were  scored  by 
one  of  the  authors  of  the  scale  and  four  of  his  associates  and  a  score 
assigned  for  each  of  the  elements  of  form,  movement  and  spacing. 
The  specimens  were  coded  so  that  the  students  were  unaware  of  the 
''expert"  score. 

In  scoring  these  practice  samples,  and  subsequently  the  test 
papers,  the  ascending-descending  method  was  used. 

It  was  planned  to  give  the  student  judges  an  initial  test  in 
scoring  and  at  the  end  of  the  practice  period  a  final  test.  The  data 
resulting  from  these  tests  would  afford  a  measure  of  the  reliability 
of  each  prospective  judge  and  a  basis  for  eliminating  some  of  the 
students  from  participation  in  scoring  the  test  papers.  Unfortu- 
nately school  conditions  and  other  circumstances  prevented  us  from 
carrying  out  our  programme  in  its  entirety.  We  were,  however, 
able  to  get  an  initial  test  for  all  the  student  judges  and  a  final  test 
for  part  of  the  group. 

Table  XIV  shows  for  the  initial  test  the  average  deviation  per 
specimen  from  "expert"  scores  for  the  group  of  prospectve  judges. 
In  this  preliminary  test  there  were  twenty-one  samples  used. 


TABLE   XIV— AVERAGE  DEVIATION  PER  SPECIMEN  FROM   "EXPERT"   SCORES 
FOR  86  PROSPECTIVE  JUDGES— PRACTICE  SAMPLES 


Deviation  in  Points 

Aver- 

Element 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

age 

Form 

3 

3 

8 

24 

20 

15 

11 

2 

1 

1 

1 

5-07 

Move- 

ment- 

2 

2 

4 

18 

22 

19 

12 

5 

3 

1 

1 

6.5 

Spacing . 

1 

8 

22 

36 

12 

8 

1 

1 

5.05 

Total 

5 

4 

18 

50 

74 

49 

38 

15 

6 

4 

2 

.... 

2 

5.5 

It  developed  upon  later  analysis  of  these  results  that  for  those 
of  the  trial  specimens  which  were  taken  from  the  scale  there  was  a 
possibility  of  identifying  the  specimens  through  the  content,  which 
factor  may  have  influenced  the  variability  of  the  judging.  Accord- 
ingly, those  trial  specimens  which  had  been  taken  from  the  scale 
were  excluded  and  the  judgments  retabulated  on  the  basis  of  the 
remaining  specimens  with  the  following  results : 
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TABLE    XV— AVERAGE    DEVIATIONS    PER    SPECIMEN    FOR    "EXPERT"    SCORES 
FOR  86  PROSPECTIVE  JUDGES— PRACTICE   SAMPLES— SECOND  TABULATION 


AvEBAQB  Deviations 

Av- 

Element 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

er- 
age 

1 

4 

8 

3 
11 

16 

8 
24 

12 

3 

20 

25 

14 
18 

14 

15 

1 

2 

15 
5 

1 

17 
2 

1 

3 
2 

3 

1 

1 
1 

1 

1 

2 

7  P 

Move- 
ment  

10  1 

Spacing.. 

1 

2 

7.4 

Total 

1 

1 

6 

22 

48 

35 

57 

30 

22 

20 

6 

3 

1 

2 

1 

3 

8.3 

The  above  table  shows  the  average  deviation  of  each  of  the  86 
prospective  judges  in  scoring  the  practice  samples.  Under  form,  for 
instance,  one  judge  deviated  on  the  average  from  1  to  2  points  from 
the  "standard"  scores  given  the  practice  samples,  4  deviated  on  the 
average  4  points,  8  deviated  5  points,  and  so  on.  The  average  of 
these  average  deviations  was  7.9  for  form,  10.1  for  movement  and 
7.4  for  spacing.  The  greatest  deviation  appeared  in  the  element  of 
movement. 


TABLE   XVI— DEVIATIONS  PER  JUDGMENT 


Element 

^0 

35 

30 

25 

20 

15 

10 

5 

0 

+5 

10 

15 

20 

25 

30 

35 

40 

Total 

Aver. 

Form .. 

2 
22 

1 

11 

20 

2 

12 

38 

4 

26 
89 
16 

46 
51 
76 

134 
211 

97 

134 
168 
226 

300 
171 
216 

154 
206 
317 

156 

28 
118 

78 
85 
86 

35 

5 

17 

12 
20 
16 

12 
2 
3 

1 

4 
2 

1 
1 

1,114 
1,125 
1,199 

7  9 

Movement.... 
Spacing. 

5 

1 

10.1 
7.2 

Total 

6 

25 

33 

54 

131 

173 

442 

528 

687 

677 

302 

249 

57 

48 

17 

7 

2 

3,438 

8.4 

Table  XVI  shows  the  distribution  of  the  judgments  given  by  the 
86  prospective  judges  in  scoring  the  practice  samples,  in  terms  of  their 
deviations  plus  and  minus  from  the  standard  or  expert  scores.  The 
table  is  read  as  follows : 

Out  of  1,114  judgments  in  form  given  by  the  86  judges  2  were  35 
points  below  the  "expert"  score,  11  fell  30  points  below,  12,  25  points 
below,  and  so  on.  Three  hundred  judgments  agreed  with  the  "expert" 
score,  154  exceeded  it  by  5  points,  156  by  10  points  and  so  on.  Like- 
wise for  the  other  elements. 

Table  XVII  shows  for  each  element  the  nimiber  and  percent,  of 
judgments  that  deviated  by  various  scale  amounts. 
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TABLE   XVII— DEVIATION  ACCORDING  TO  AMOUNT 


Deviations 

FOHM 

Movement 

Spacing 

Total 

No. 

Percent. 

No. 

Percent. 

No. 

Percent. 

No. 

Percent. 

Zero  deviations — 

5  points  or  \^  step 
10  points  or  1  step.. 
15  points  or  ij^  step 
20  or  2  steps  or  more 

300 
288 
290 
124 
112 

26.9 
25,9 
26.0 
11.1 
10.0 

171 
374 
239 
136 
205 

15.2 
33.2 
21.2 
12.1 

18.2 

216 
643 
215 
162 
63 

18.0 
45.3 
18.0 
13.5 
6.2 

687 
1,205 
744 
422 
380 

20.0 
35.1 
21.6 
12.3 
11.0 

Total 

1,114 

99.9 

1,125 

99.9 

1,199 

100.0 

3,438 

100.0 

Under  form  it  v^ill  be  seen  that  878  or  78.8  percent,  of  the  judg- 
ments fell  within  one  step  of  the  scale  from  the  "standard  or  expert" 
score.  In  movement  69.6  percent,  of  the  judgments  deviated  one 
step  or  less,  while  in  spacing  the  number  of  such  judgments  was  81.3 
percent. 

It  is  to  be  borne  in  mind  that  the  above  data  are  for  the  initial 
test  given  at  the  beginning  of  the  period  during  which  it  was  planned 
to  give  rather  intensive  training  to  those  students  who  were  to  serve 
as  scorers.  While  school  conditions  and  extraneous  circumstances 
prevented  us  from  carrying  out  our  programme  in  its  entirety,  it  is 
believed  that  the  variability  of  those  who  finally  acted  as  judges  was 
considerably  reduced  from  the  degree  indicated  in  the  above  table. 

For  a  number  of  the  judges  to  whom  a  second  formal  test  was 
given  it  was  possible  to  secure  data  which  indicated  the  improvement 
attained.  Table  XVIII  shows  for  56  of  our  prospective  judges  the 
average  deviations  in  the  initial  test  in  comparison  with  the  results 
of  the  final  test. 

From  this  table  it  is  seen  that  the  average  deviation  for  the  56 
judges  on  the  initial  test  was  7.5  points  on  the  scale  or  ^  of  a  step. 
On  the  second  test,  several  weeks  later,  the  average  deviation  of  the 
group  was  4.2  points  or  less  than  3^  of  a  scale  step.  The  figures  to 
the  left  of  the  heavy  zigzag  line  indicate  the  students  who  showed 
improvement  in  scoring.  Those  to  the  right  of  the  line  represent 
the  students  whose  scoring  showed  the  same  amount  of  deviation  or 
an  increase  therein.  For  instance,  there  were  17  students  whose 
average  deviation  on  the  first  test  was  8  points.  On  the  second 
test  these  17  students  were  distributed  as  follows:  1  deviated  less 
than  one  point;  1  deviated  one  point;  5  deviated  two  points;  2, 
six  points;  and  1,  seven  points.  All  these  showed  considerable 
improvement.  On  the  other  hand,  4  deviated  nine  points  or  one 
point  more  on  the  second  test  than  on  the  first  test;  1,  ten  points 
and  1,  eleven  points.     These  six  showed  a  loss  in  accuracy.     Of  the 
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56  students,  42  showed  improvement,  in  some  cases  marked,  3  showed 
no  progress,  while  11  showed  a  sHght  loss,  that  is  an  increase  in  varia- 
bility. 

TABLE  XVIII— AVERAGE   DEVIATIONS  FOR  56  PROSPECTIVE   JUDGES   IN 

TWO  TESTS 


Average  Deviations — 2d  Test 

Total 

Devia- 
tions 

0-1 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

n 

12 

13 

14 

15 

16 

17 

18 

1st 
Test 

18 

1 

1 

17 

1 

1 

16     

15    

14 

13 

12 

1 

1 

11  

1 

1 

10        

1 

1 
1 

2 

1 

2 
2 

[l 

2 

1 

4 
1 

1 

9 

2 

1 
1 
2 
2 

1 
1 

5 

3 

1 
1 

1 

1 

3 

1 

.1 

1 
1 

8 

8 

1 

17 

7 

6 

6  ...._ 

11 

5 

6 

4 

2 

3  

2  

1      . 

1 

1 

Totals 
2nd  Test.... 

8 

3 

10 

7 

4 

4 

5 

2 

4 

5 

2 

1 

1 

56 

Average  4.5 

Aver. 
7.5 

.^l) 


APPENDIX  B 

Reliability  of  the  Scoring 

The  papers  of  about  15  percent,  of  the  classes  involved  in  the 
test  were  scored  by  groups  of  three  judges,  each  scorer  working 
independently.  The  data  from  twenty  such  classes  were  ana- 
lyzed and  afford  a  fair  measure  of  the  reliability  of  the  scoring 
in  general. 

Table  XIX  shows  as  an  illustration  the  scores  in  form  assigned 
by  three  judges  to  each  paper  in  a  5 A  class.  The  table  is  to  be  read 
as  follows: — ^Judge  I  rated  the  paper  of  pupil  number  1,  at  30  on  the 
scale  in  form,  Judge  II  rated  it  as  worth  25,  and  Judge  III  assigned 
the  mark  of  30.  The  median  of  the  three  judgments  was  30.  The 
•  scores  of  Judge  I  and  III  showed  zero  deviation,  while  that  of  Judge 

II  was  5  points  below.  The  total  deviation  of  the  group  from  the 
median  score  was  5  points,  or  an  average  of  1.7  points.  Out  of  the 
total  of  forty  specimens  in  the  class,  the  three  judges  agreed  absolutely 
as  to  the  value  of  13,  or  about  one-third  of  the  papers.  On  most  of 
the  other  samples  also  the  judges  showed  very  close  agreement.  In 
only  a  few  cases  is  there  any  wide  variation,  for  instance,  papers 
No.  7  and  18. 

At  the  bottom  of  the  columns  headed  "Deviations"  appears  the 
simi  of  the  deviations  of  the  scores  of  each  judge.  Judge  I  shows  a 
total  of  70  on  the  40  papers,  or  an  average  deviation  of  1.8  points; 
Judge  II  has  a  total  of  130  or  an  average  deviation  of  3.3  points,  while 
Judge  III  deviated  on  the  average  only  four  tenths  of  a  point.  The 
total  deviation  on  the  entire  set  of  papers  for  the  three  scorers 
was  215  or  an  average  of  1.8  points,  less  than  a  fifth  of  a  step 
on  the  scale. 

At  the  foot  of  the  columns  headed  ''scores"  appears  the  medians 
based  upon  the  scores  given  by  each  judge.  The  median  of  the 
scores  given  by  Judge  I  is  computed  at  39.5,  that  of  Judge  II  at40.7, 
and  that  of  Judge  III  at  39.4.  The  median  class  score  computed 
from  the  median  score  on  each  sample  is  40.  The  class  medians  given 
by  the  three  judges  are  practically  identical,  Judge  I  differing  but 
by  five  tenths  of  a  point,  Judge  II  by  only  seven  tenths  and  Judge 

III  by  six  tenths  of  a  point.  The  average  deviation  of  class  medians 
for  the  three  judges  from  the  class  median  computed  from  the  median 
score  on  each  sample  is  but  six  tenths  of  a  point. 
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TABLE  XIX— SCORES  IN  FORM  OF  A  FIFTH  GRADE  CLASS  GIVEN  BY 
THREE  JUDGES  SCORING  INDEPENDENTLY 


Pupil's 

scohb  accobding  to 
Judge 

Median 
Score 

Deviation  op 
Judge 

Each 

Total 

Average 

Sample 

I 

II 

III 

I 

II 

III 

Dev. 

1  

30 
30 
35 
40 
40 
40 
45 
35 
35 
25 
35 
50 
40 
40 
30 
35 
30 
30 
40 
40 
30 
40 
35 
35 
30 
40 
30 
40 
45 
45 
40 
30 
40 
40 
25 
40 
35 
35 
40 
35 

25 
35 
40 
40 
40 
45 
70 
35 
25 
25 
25 
60 
35 
35 
25 
25 
25 
45 
35 
50 
40 
50 
25 
40 
25 
45 
30 
40 
45 
40 
40 
40 
40 
40 
25 
40 
35 
35 
40 
40 

30 
35 
35 
40 
40 
35 
50 
35 
25 
25 
35 
60 
40 
40 
25 
35 
25 
30 
35 
40 
30 
40 
30 
40 
30 
45 
30 
40 
45 
45 
40 
40 
40 
.  45 
25 
40 
35 
30 
40 
35 

30 
35 
35 
40 
40 
40 
50 
35 
25 
25 
35 
60 
40 
40 
25 
35 
25 
30 
35 
40 
30 
40 
30 
40 
30 
45 
30 
40 
45 
45 
40 
40 
40 
40 
25 
40 
35 
35 
40 
35 

0 

—5 
0 
0 
0 
0 

—5 
0 
+10 
0 
0 
—10 
0 
0 

+5 
0 

+5 
0 

+5 
0 
0 
0 

+5 

—5 
0 

—5 
0 
0 
0 
0 
0 
—10 
0 
0 
0 
0 
0 
0 
0 
0 

—5 
0 

+5 
0 
0 

+5 
+20 
0 
0 
0 
—10 
0 

—5 

—5 

0 

—10 

0 

+15 

0 

+  10 

+10 

+10 

—5 
0 

—5 
0 
0 
0 
0 

— 5 
0 
0 
0 
0 
0 
0 
0 
0 
0 

+5 

0 
0 
0 
0 
0 

—5 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0. 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

+5 
0 
0 
0 

—5 
0 
0 

5 

5 

5 

0 

0 

10 

25 

0 

10 

0 

10 

10 

5 

5 

5 

10 

5 

15 

5 

10 

10 

10 

10 

5 

5 

5 

0 

0 

0 

5 

0 

10 

0 

5 

0 

0 

0 

5 

0 

5 

1.7 

2 

3.. 

1.7 
1.7 

4 

0.0 

5 

0.0 

6 

7... 

3.3 

8.3 

8 

0.0 

9 

3.3 

10 

0.0 

11 

3.3 

12.. 

3.3 

13. 

1.7 

14. 

1.7 

15.. 

1.7 

16 

3.3 

17 

1.7 

18 

5.0 

19 

1.7 

20. 

3.3 

21 

3.3 

22..  . 

3.3 

23.. 

3.3 

24.. 

1.7 

25.. 

1.7 

26. 

1.7 

27 

28 

0.0 
0.0 

29... 

0.0 

30.. 

1.7 

31  ..  .. 

0.0 

32.. 

3.3 

33.. 

0.0 

34.. 

1.7 

35.. 

0.0 

36. 

0.0 

37-. 

0.0 

38 

1.7 

39 

0.0 

40..-.. 

1.7 

Class 
median 

39.5 

40.7 

39.4 

40 

Totals 

70 

130 

15 

215 

Average 

1.8 

3.3 

0.38 

1.8 

This  is  remarkably  close  scoring.  It  is  probably  the  best  per- 
formance revealed  by  the  analysis  of  the  data.  The  following  tables 
will  show  that  while  not  as  excellent  a  showing  was  made  by  the 
other  scorers,  the  scoring  was  on  the  whole  very  consistent  and 


quite  satisfactory. 
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Figure  XXVII 
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The  twenty  classes  for  which  the  data  were  worked  up  in  detail 
totaled  753  specimens.  Each  of  these  specimens  received  three 
scorings  or  judgments,  making  a  total  of  2,259  judgments.     The  fol- 
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lowing  table  and  figures  show  the  distribution  of  these  judgments  in 
terms  of  deviations  from  the  median  score  for  each  paper. 


TABLE  XX— DISTRIBUTION  OF  INDIVIDUAL  SCORES  IN  TERMS  OF  DEVIA- 
TIONS FROM   MEDIAN  SCORES  ON  EACH  SAMPLE 


Deviations 

Total 

—30 

—25 

—20 

—15 

—10 

—5 

0 

+  5 

+10 

+15 

+20 

+25 

+30 

age 

Form 

1 

1 
2 

2 
5 

4 

11 
12 
16 

37 
55 
64 

128 
137 
143 

297 

281 
249 

1,310 
1,249 

1,279 

278 
300 
288 

131 
152 
140 

49 
42 
56 

10 
18 

12 

2 
6 
6 

3 

1 

2,2.59 
2,259 
2,259 

3  27 

Movement.— 
Spacing— 

3..62 
3..62 

Total- 

4 

11 

39 

156 

408 

827 

3,838 

866 

423 

147 

40 

14 

4 

6,777 

3..5 

Form 

Movement 

Spacing 

Number 

Percent. 

Number 

Percent. 

Number 

Percent. 

0 

1,310 
575 
259 
115 

58..0 

25..4 

11..5 

5..1 

1,249 
581 

2S9 
140 

55..3 

25..7 

12..8 

6.-2 

1,279 
537 
283 
160 

56..6 

23-.8 

10  points  or  1  step 
More  than  10  poin 

12  5 

s  or  1  step..-,. 

7..1 

Total 

2.259 

100..0 

2.259 

100..0 

2.259 

100-0 

1 

Out  of  the  total  of  2,259  judgments  in  form,  1,310  or  58.0  per- 
cent, agreed  with  the  median  scorej  473  or  20.9  percent,  were  posi- 
tive deviations  and  476  or  21.0  percent,  were  negative.  The  lower 
part  of  the  table  shows  that  25.4  percent,  of  the  judgments  deviated 
5  points  or  one  half  step  on  the  scale;  11.5  percent,  deviated  one 
step  and  5.1  percent,  more  than  one  step. 

In  movement  55.3  percent,  of  the  judgments  agreed  with  the 
median  scores,  519  or  23.0  percent,  were  positive  deviations,  while 
491  or  21.7  percent,  were  negative. 

Out  of  the  total,  25.7  percent,  of  the  judgments  deviated  one 
half  step  on  the  scale,  12.8  percent,  deviated  a  full  step  and  6.2  per- 
cent, more  than  a  full  step. 

In  spacing,  1,279  out  of  the  2,259  judgments  or  56.6  percent, 
agreed  with  the  median  scores,  502  or  22.2  percent,  were  positive 
deviations  while  478  or  21.2  percent,  were  negative.  Of  the  total 
judgments,  23.8  percent,  deviated  5  points  or  half  a  scale  step,  12.5 
percent,  deviated  a  full  step  and  7.1  percent,  more  than  a  scale 
step. 

The  average  deviation  per  judgment  is  3.3  points  in  form,  3.6  in 
movement  and  3.6  in  spacing.     This  means  that  on  the  average 
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a  judgment  by  one  judge  on  an  individual  sample  will  vary  less  than 
four  tenths  of  a  scale  step  from  the  median  of  three  independent 
judgments. 

Looking  at  the  matter  from  the  standpoint  of  the  individual 
judge  rather  than  the  individual  paper,  we  may  consider  the  data 
presented  in  Table  XXI.  Table  XXI  shows  for  each  of  the  sixty 
judges  involved  the  average  deviation  per  sample.  These  aver- 
age deviations  are  based  on  the  judgments  given  to  the  papers  of 
a  class  compafed  with  the  median  score  of  the  group  of  three  judges 
in  each  case. 

TABLE  XXI— AVERAGE  DEVIATION  PER  SAMPLE  FOR  60  JUDGES 


Amount 

0.0  to     1  point 

1  to    2  points -. 

2  to    3  points 

3  to    4  points 

4  to    5  points.- 

5  to    6  points.- 

6  to    7  points 

7  to    S  points 

8  to    9  points. 

9  to  10  points 

Total. 

Average _ 


Form 


Movement 


Spacing 


17 

16 

11 

2 

3 


60 
3.3 


1 

7 

13 

19 

11 

4 

2 

1 

1 


60 
3.6 


17 
10 
9 
6 
1 
3 
1 


60 
3.7 


For  instance,  two  out  of  the  sixty  judges  showed  an  average 
deviation  on  the  papers  of  the  classes  scored  by  them  of  less  than 
one  point;  eight  judges  differed  less  than  two  points;  seventeen 
judges  deviated  on  the  average  less  than  three  points,  and  so  on. 

In  form  the  average  of  the  average  deviations  for  the  group  of 
sixty  judges  was  3.3  points  per  judge,  in  movement  it  was  3.6  points, 
and  in  spacing  3.7  points. 

While  the  scoring  on  the  whole  showed  very  satisfactory  agree- 
ment, there  was  considerable  variation  shown  by  individual  judges 
in  the  case  of  individual  samples.  The  bulk  of  the  papers  were 
scored  by  a  single  judge.  In  order  to  get  a  true  index  of  the  value  of 
any  individual  paper  it  would  probably  be  necessary  to  have  such 
paper  scored  by  more  than  one  judge  or  more  than  once  by  the  same 
judge.  With  an  increase  in  the  number  of  judges  we  would  obtain  a 
consensus  of  opinion  which  would  yield  us  the  real  value  of  the  paper. 
Otu  study,  however,  was  directed  more  to  the  performance  or  achieve- 
ments of  groups  of  pupils  rather  than  of  individuals. 

In  dealing  with  the  class  scores  it  is  probably  safe  to  say  that 
such  scores  are  more  reliable  than  any  score  given  to  an  individual 
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paper.  On  the  whole,  while  the  scores  of  an  individual  judge  may 
vary  more  or  less  widely  from  the  median  of  the  three  independent 
judgments,  the  median  score  for  a  class  set  of  papers  given  by  one 
judge  shows  less  variation  and  closer  agreement  with  the  class  median 
based  on  the  group  scores.  It  is  possible  and  probable  that  the  devi- 
tions  of  the  individual  scores  for  the  one  judge  may  counteract  each 
other, 

Figure  28  shows  graphically  the  data  presented  in  Table  XX,  the 
distribution  of  the  2,259  judgments  in  terms  of  their  plus  or  minus 
deviations.  In  each  of  the  elements  the  S3nimietrical  character  of 
the  diagrams  is  to  be  noted.  The  number  of  the  positive  deviations 
equals  or  nearly  equals  the  number  of  negative  deviations.  In  the 
scoring  of  large  groups  of  papers,  the  deviations  in  one  direction 
would  in  all  probability  be  corrected  or  counteracted  by  the  devia- 
tions in  the  other  directions  so  that  the  median  as  fotmd  would 
approximate  very  closely  the  actual  median  value  of  the  group 
performance. 
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TABLE   XXII— AVERAGE    DEVIATION   PER   CLASS   SCORE    FOR   TWENTY 
GROUPS  OF  SCORERS 


Amount 

0.0  to  1.0  point 

1  to  2  points 

2  to  3  points 

3  to  4  points._ 

4  to  5  points.- 

5  to  6  points 

Total 

Average 


Form 


Movement 


Spacing 


20 
2.0 


20 

2.4 


20 

2.S 


Table  XXII  shows  the  average  deviation  of  each  group  of  three 
judges  from  the  class  median  based  on  the  group  scores.  In  form 
out  of  the  20  groups,  5  groups  deviated  on  the  average  loss  than  1 
point,  7  less  than  2  points,  3  less  than  three  and  1  more  than  four 
points.  The  average  of  the  average  deviations  was  2  points  in  form, 
2.4  points  in  movement  and  2.8  points  in  spacing.  For  the  individual 
judges  rather  than  for  groups,  the  average  of  the  average  deviation  in 
form  was  2.2  points,  in  movement  2.6  points  and  in  spacing  2.7 
points. 

On  the  whole,  then,  the  class  medians  resulting  from  the  scoring 
of  one  judge  may  be  taken  as  the  actual  value  of  the  writing  of  the 
class  and  will  be  in  error  less  than  three  points,  or  three  tenths  of  a 
scale  step. 

Our  main  interest  lies  not  even  in  the  performance  of  class  groups 
but  in  the  achievement  of  larger  grade  groups.  With  the  increase  in 
numbers  involved  there  is  a  strong  probability  that  the  grade  medians 
given  for  the  entire  group,  involving  in  each  case  about  1,000  pupils, 
represents  the  actual  value  of  the  performance  of  the  group. 
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